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.•  ^  image  is  stored  or  transmitted,  we  have  access  to  the  original  and  the  distorted 

versio^.  The  enhanced  codec  is  compared  to  the  original  block  by  block  to  determine  which  blocks 
have  teen  improved  by  the  enhancement  These  blocks  are  then  flagged  for  post-processing  in  a 
way  that  IS  compliant  with  the  JPEG  standard  and  adds  nothing  to  the  compressed  image’s 
bandwidth.  The  end  result  is  a  compressed  image  that  can  be  decompressed  on  any  standaid  JPEG 
decompressor,  but  that  can  be  enhanced  by  a  sophisticated  decompressor. 

For  Ae  comparison  of  the  original  and  enhanced  images,  we  have  teen  developing  a  new 
mteel  that  is  specifically  tailored  to  the  detection  of  errors  that  occur  within  or  between  two 
JPEG  codec  blocks.  Previous  filter  models  have  been  restricted  from  using  a  large  number  of  filters 
due  to  computational  constraints  which  we  avoid  by  focusing  the  model  on  a  tiny  spatial  area  of 
8x16  pixels.  Further,- features  of  human  vision  that  have  teen  included  in  previous  models  (color 
temporal,  stereo  etc.)  are  not  needed  for  this  more  focused  problem.  Issues  that  have  not  teen 
completely  addressed  by  previous  models,  such  as  masking  effects,  are  tractable  and  the  model  is 
more  applicable  to  JPEG  compression. 
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Image  Enhancement  Vision  Modeling  Image  Compression  JPEG 
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Final  Progress  Report 

1.  Principal  Investigator 

Prof.  Stanley  A  Klein  (510)  643  8670 

University  of  California  at  Berkeley 
School  of  Optometry 
Berkeley  CA  94720 

Grant  Number:  F49620-92-J-0359 

2.  Objectives 

As  previously  described 

3.  Status  of  the  effort  . 

We  have  developed  a  system  for  compressmg  and  restoring  images  that  is 
compatible  with  the  JPEG  picture  compression  s^dard.  Direct  application  of  ^ditional 
image  enhancing  techniques  fails  to  utilize  a  unique  feature  that  can  be  exploited  when 
working  with  digital  coinprcssiont  before  a  JPEG  image  is  stored  or  transmitted,  we  have 
access  to  the  original  (figure  A)  and  the  distorted  versions  (figure  C).  Enhaiicement  can 
be  selectively  applied  and  assessed,  and  successful  enhancement  instiiictions  can  be 
included  with  the  image  file.  Image  blocks  are  flagged  for  post-processing  widi  a  par^ 
adjusting  system  that  is  compliant  with  the  JPEG  standard  and  adds  nothing  to  the 
compressed  images’  bandwidth.  The  end  result  is  a  compressed  image  that  can  be 
decompressed  on  any  standard  JPEG  decompressor  (figure  D),  but  that  can  be  enhanced 
by  a  sophisticated  decompressor  (figure  B)  that  makes  use  of  the  hidden  flags  . 

To  assess  the  success  of  the  enhancement  instructions,  we  have  developed  a  new 
model  of  human  vision.  Our  model  is  particularly  well  suited  to  detecting  errors  inside 
and  between  JPEG  image  blocks.  Previous  filter  models  have  been  restricted  from  using  a 
large  number  of  filters  due  to  computational  constraints,  which  we  have  avoided  by 
focusing  the  model  on  a  tiny  spatial  area  of  8x16  pixels. 


4.  Accomplishments  /  New  Findings 

The  Regents  of  the  University  of  California  have  applied  for  a  U.S.  Patent  on  the 
invention  and  are  seeking  licensees  to  commercialize  this  methodology.  The  model  has 
broad  applications  in  image  quality  assessment  as  well. 

In  addition  to  the  research,  we  developed  computer  software  for  psychophysical  testing 
and  vision  model  development  that  will  be  useful  for  other  researchers.  Amnon  ^ 
Silverstein  is  presently  preparing  this  software  for  distribution  to  other  laboratories. 

5.  Personnel  Supported 

Amnon  Silverstein 

6.  Publications 

D.  A.  Silverstein,  S.  A.  Klein  “Precomputing  and  encoding  compressed  image 
enhancement  instractions”  [In  review  for  IEEE  transactions  on  image  processmg] 

D.  A.  Silverstein,  “A  DCT  image  fidelity  metric  and  its  appUcation  to  a  text-based 
scheme  for  image  display”  SPIE  Human  Vision,  Visual  Processing,  and  Digital  Display 
TTT,  Ed,  Rogowitz  &  Allebach  Vol.1913,  pp.229-239,1993 


D.  A.  Silverstein,  S.  A.  Klein  “Restoration  of  compressed  images”  SPIE  Image  and 
Video  Compression  Vol.  2186, 56-64, 1994 

S.  A.  Klein,  D.  A.  Silverstein  &  T.  Carney  “Relevance  of  human  vision  to  JPEG-DCT 
compression”  SPIE  Human  Vision,  Visud  Processing,  and  Digital  Display  HI,  Ed, 
Rogowitz  &  Allebach  Vol.  1666, 200-215, 1992 

7.  Interactions/Transitions 

Our  new  vision  model  was  presented  at  the  ARVO  conference  in  Sarasota,  May  17  1995. 
Abstract  2142-109,  “Can  an  Oriented-Filter  Model  Predict  the  Detectability  of  Spatially 
Local  Stimuli?” 

Our  image  compression  and  enhancement  work  was  presented  at  the  SPIE  conference  in 
San  Jose,  1994  “Restoration  of  compressed  images” 

Both  of  these  topics  were  presented  at  a  talk  in  Hewlett  Packard  Laboratories  in  Palo 
Alto,  July  1995  to  Dr.  Joyce  Farrell’s  Photographic  Image  Quality  group  and  the  Image 
Technology  research  group. 

Amnon  Silverstein  became  an  employee  at  Hewlett  Packard  Labs  in  Palo  Alto  August  1, 
1995.  He  has  been  incorporating  Ae  knowledge  and  techniques  developed  from  his  work 
in  this  project  to  picture  quality  and  printing  issues.  He  is  further  developing  the  vision 
model  for  application  to  halftone  design  and  assessment.  He  is  also  designing  new 
methods  for  utilizing  JPEG  image  compression  for  printed  images. 

8.  Patent  Disclosure 

We  have  filed  a  disclosure  for  a  patent  through  the  University  Office  of  Technology 
Licensing,  Attomy’s  Docket  No.  028726-009  “Precomputing  and  encoding  compressed 
image  enhancement  instructions” 
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